As the use of digital electronic communication networks has grown in recent years, the sophistication of internal and external network attacks in the form of viruses, Trojan horses, worms, and malware of various kinds has increased dramatically. Just as dramatic is the accelerated increase of network speeds and a corresponding drop in network costs, thereby driving the rapid adoption of networks. These and other factors have necessitated the development of innovative and more advanced network security measures.
For example, Intrusion Detection Systems (IDS) can often detect network attacks, but as passive systems they generally offer little more than after-the-fact notification. In contrast, Intrusion Prevention Systems (IPS) have been developed to complement traditional security products, such as firewalls, by proactively analyzing network traffic flows and active connections while scanning incoming and outgoing requests. As network traffic passes through the IPS, it is examined for malicious packets. Such examination may be performed by one or more “deep packet inspection engines” which perform “deep packet inspection” on some or all of the packets in the network traffic. Traffic is blocked if the IPS identifies it as posing a potential threat or as being associated with an unwanted application, while legitimate traffic is allowed to pass through the system unimpeded.
Properly implemented, an IPS can be an effective network security safeguard. There are, however, needs for improved IPS capabilities. For example, an IPS may include multiple deep packet inspection engines for performing deep packet inspection on traffic flows passing through the IPS because a single deep packet inspection engine, typically implemented as a microprocessor executing a suitable operating system and software, may not be capable of processing the flows at a sufficiently high throughput. Techniques for balancing network traffic load among multiple deep packet inspection engines in an IPS to increase the aggregate performance of such engines and thereby the overall performance of the IPS are disclosed in U.S. patent application Ser. No. 11/443,490, filed by Brian C. Smith, Alexander Sarin, and Hazem M. Kadaba on May 30, 2006, entitled “Intrusion Prevention System Edge Controller”; and U.S. patent application Ser. No. 11/782,840, filed by Gerald S. Stellenberg, Brian C. Smith, and James M. Rollette on Jul. 25, 2007, entitled “System and Method for Traffic Load Balancing to Manage Muliple Processors”.
When using multiple deep packet inspection engines, where each engine is assigned multiple communication sessions and where traffic shaping is configured to pace the traffic of a specified protocol type (e.g., HTTP traffic), and that controlled traffic is spread across the multiple engines, there is a need to coordinate the output of the multiple engines, such that the aggregate rate of the output from the multiple engines for each protocol type is limited to the specified allowed rate for that protocol type. In other words, it is desirable to enforce a system wide rate limit for traffic of a particular protocol type even when traffic of that protocol type is spread across multiple engines.
What is needed, therefore, are techniques for managing traffic flow processing in an IPS or other network device to distribute available bandwidth among traffic flows.